feat: add no-ai filtering support for my feed by nimrodkra · Pull Request #3766 · dailydotdev/daily-api

nimrodkra · 2026-04-01T08:16:46Z

Summary

add validated AI tag and title-word blocklists for the No-AI experiment
extend the feed query and My Feed resolver to merge No-AI filters without changing saved feed settings
make the local/v1 path honor title-word blocking so dev behavior is closer to production

Test plan

pnpm run lint
pnpm run build

Made with Cursor

Merge curated AI tag and title-word filters into My Feed requests so the frontend can ship the experiment without persisting user feed settings. Made-with: Cursor

pulumi · 2026-04-01T08:25:10Z

🍹 The Update (preview) for dailydotdev/api/prod (at e805896) was successful.

✨ Neo Explanation

This is a standard application deployment rolling out a new build to all 7 services and 37 scheduled jobs, accompanied by database and ClickHouse schema migration jobs that clean up the previous release's migration jobs and create new ones for this version.

Root Cause Analysis

A new version of the API application has been built and is being deployed to production. Every workload in the cluster references the container image by version tag, so publishing a new image triggers updates across all Deployments, CronJobs, and migration Jobs simultaneously.

Dependency Chain

The new container image version cascades uniformly across the entire fleet:

7 Deployments (main API, background worker, WebSocket, private, temporal, personalized digest, worker jobs) are all rolling out the new image
37 CronJobs are updated to reference the new image version in their job templates — these will take effect on their next scheduled run
2 one-time migration Jobs (database and ClickHouse) are being replaced: the old versions tied to the previous release are deleted, and new ones for the current release are created to run any schema migrations required by this code change

Risk analysis

No stateful resources (databases, storage buckets, persistent volumes) are being replaced or deleted. The migration Jobs run schema changes against the database and ClickHouse, which carries inherent risk if the migrations are destructive or irreversible, but this is standard practice for this deployment pattern. Deployments use rolling updates by default in Kubernetes, so no downtime is expected for the running services.

Resource Changes

    Name                                                       Type                           Operation
~   vpc-native-update-current-streak-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-analytics-history-clickhouse-cron  kubernetes:batch/v1:CronJob    update
~   vpc-native-update-source-public-threshold-cron             kubernetes:batch/v1:CronJob    update
~   vpc-native-validate-active-users-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-update-trending-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-rotate-daily-quests-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-update-achievement-rarity-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-rotate-weekly-quests-cron                       kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-stale-user-transactions-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-deployment                                      kubernetes:apps/v1:Deployment  update
~   vpc-native-channel-digests-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-opportunities-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-generic-referral-reminder-cron                  kubernetes:batch/v1:CronJob    update
+   vpc-native-api-db-migration-af9ea149                       kubernetes:batch/v1:Job        create
+   vpc-native-api-clickhouse-migration-af9ea149               kubernetes:batch/v1:Job        create
~   vpc-native-user-posts-analytics-refresh-cron               kubernetes:batch/v1:CronJob    update
~   vpc-native-ws-deployment                                   kubernetes:apps/v1:Deployment  update
~   vpc-native-update-tag-materialized-views-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-private-deployment                              kubernetes:apps/v1:Deployment  update
~   vpc-native-post-analytics-clickhouse-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-users-cron                         kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-analytics-clickhouse-cron          kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-updated-sync-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-update-highlighted-views-cron                   kubernetes:batch/v1:CronJob    update
-   vpc-native-api-clickhouse-migration-e28a81e8               kubernetes:batch/v1:Job        delete
~   vpc-native-worker-job-deployment                           kubernetes:apps/v1:Deployment  update
~   vpc-native-expire-super-agent-trial-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-sync-subscription-with-cio-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-channel-highlights-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-calculate-top-readers-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-squad-posts-analytics-refresh-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-expired-better-auth-sessions-cron         kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-deployment                  kubernetes:apps/v1:Deployment  update
~   vpc-native-clean-zombie-user-companies-cron                kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-images-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-hourly-notification-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-update-tags-str-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-channel-highlights-cron                         kubernetes:batch/v1:CronJob    update
-   vpc-native-api-db-migration-e28a81e8                       kubernetes:batch/v1:Job        delete
~   vpc-native-update-views-cron                               kubernetes:batch/v1:CronJob    update
... and 9 other changes

nimrodkra · 2026-04-01T08:32:07Z

@claude please review this pr

claude · 2026-04-01T08:32:25Z

Claude finished @nimrodkra's task in 2m 45s —— View job

PR Review

Gathering context
Reading changed files
Analyzing code
Posting review

The overall approach is clean — the withConfigTransform pattern on FeedGenerator is a nice abstraction, and correctly separating v1 (local SQL path) from v2+ (feed service path) is the right way to handle both cases. A few issues worth addressing before merging:

Issues

🔴 Performance — 82 individual ILIKE conditions on v1 path

src/common/feedGenerator.ts:709-720

For noAi=true on a v1 feed, 82 separate NOT ILIKE '%word%' WHERE clauses are added to the SQL query. PostgreSQL must evaluate each one as a full string scan on every row. For large feeds this will be a meaningful regression.

A single regex condition is dramatically cheaper:

if (filters?.blockedWords?.length) {
  const pattern = filters.blockedWords
    .map((w) => w.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'))
    .join('|');
  newBuilder = newBuilder.andWhere(
    `COALESCE(${alias}.title, '') !~* :blockedWordsPattern`,
    { blockedWordsPattern: pattern },
  );
}

Fix this →

🔴 High false-positive risk — `Cursor`, `RAG`, `inference`, `transformer`

src/common/noAiFilter.ts

Several words in NO_AI_BLOCKED_WORDS are common general programming terms:

Word	Legitimate non-AI context
`Cursor` (line 115)	CSS `cursor` property, cursor pagination, database cursors, text cursors
`RAG` (line 132)	Red-Amber-Green status in project management
`inference` (line 149)	TypeScript type inference, statistical inference, logical inference
`transformer` (line 146)	Babel/esbuild AST transformers, electrical transformers

A post titled "Custom CSS cursor effects" or "TypeScript type inference tips" would be blocked for users who want to avoid AI content. Recommend removing or scoping these to compound forms only (e.g., AI inference, RAG pipeline instead of bare RAG/inference).

🟡 False positives — person names

src/common/noAiFilter.ts:164-167

Elon Musk and Jensen Huang are primarily associated with Tesla/SpaceX and Nvidia gaming, respectively. Many non-AI articles (Nvidia GPU releases, Tesla news, SpaceX launches) mention them. Consider dropping these or being more targeted.

🟡 No tests

The PR adds non-trivial business logic (withNoAiFilters, wrapGeneratorWithNoAi, mergeUniqueStrings, the new SQL filter in anonymousFeedBuilder) but includes no tests. Per the repo's testing strategy, integration tests are strongly preferred.

At minimum, an integration test for the feed query with noAi: true asserting that posts with blocked tags/words are excluded would give confidence this works end-to-end on the v1 path.

🟡 No feature flag — any client can toggle this

This is described as a "No-AI experiment" but there's no GrowthBook flag controlling access. Any client can pass noAi: true to get the filtered feed. If this is intended as a controlled rollout/experiment, consider gating it server-side so you can measure impact without exposing it broadly.

🟡 Helper functions defined after use

src/schema/feeds.ts:1411-1438

mergeUniqueStrings, withNoAiFilters, and wrapGeneratorWithNoAi are defined after feedResolverV1 which calls withNoAiFilters. This works at runtime (the call is inside a callback that runs later), but it's confusing to read and conflicts with the convention of defining helpers before their consumers. Move these three helpers above feedResolverV1.

Fix this →

🟢 Positive notes

withConfigTransform is a clean, reusable hook on FeedGenerator that doesn't bleed experiment logic into the generator itself.
mergeUniqueStrings correctly deduplicates user preferences with the experiment blocklist, so existing user preferences are preserved.
The split between v1 (SQL) and v2+ (feed service) is handled correctly with separate code paths.

Improve the local filter path and add coverage so the experiment behaves closer to production while keeping the strict no-AI blocking posture. Made-with: Cursor

nimrodkra · 2026-04-01T08:45:37Z

Addressed the review on this branch:

replaced the v1 path's per-word NOT ILIKE chain with a single regex filter so we don't stack dozens of title scans
moved the No-AI helper functions above feedResolverV1 for readability
added an integration test covering feed(noAi: true) on the local/v1 path for both blocked tags and blocked title words

On the false-positive points: I'm intentionally keeping the broader title-word blocklist. Product direction here is to optimize for not leaking AI content even if that means occasionally filtering adjacent content. That tradeoff is deliberate for this first experiment.

Verification completed:

pnpm run lint
pnpm run build

I also attempted:

NODE_ENV=test npx jest tests/feeds.ts --testEnvironment=node --runInBand

That local test run is blocked in this environment because Redis is unavailable (ECONNREFUSED 127.0.0.1:6379).

idoshamun · 2026-04-01T09:00:14Z

    );
  }
+
+  if (filters?.blockedWords?.length) {


We dont need this here. anonymousFeed shouldn't support the ai filter anyway. We should limit the blast radius to my feed

Removed the local SQL title-word filtering and reverted the anonymous-side expansion. The No-AI experiment now stays on the supported My Feed path only.

idoshamun · 2026-04-01T09:02:14Z

      fetchQueryParams: async (ctx, args) => {
        const feedId = args.feedId || ctx.userId;
-        return feedToFilters(ctx.con, feedId, ctx.userId);
+        const filters = await feedToFilters(ctx.con, feedId, ctx.userId);


Resolver v1 is also out of scope

Reverted the v1 resolver merge as requested. noAi is now only applied on the v2 feed-service path, and the test was moved to assert the v2 request config instead.

Remove the anonymous and v1-only backend paths so the no-ai filter stays scoped to My Feed's supported feed-service flow. Made-with: Cursor

nimrodkra · 2026-04-01T09:09:56Z

Addressed idoshamun's requested scope reduction on this branch:

removed the local SQL / anonymous-side title-word filtering
reverted the v1 resolver merge
kept noAi applied only on the supported v2 feed-service path for feed
updated the test to assert the v2 request contains the injected blocked tags and title words

Verification:

pnpm run lint
pnpm run build

I also retried the feed Jest suite, but it is still blocked in this environment because local Redis is unavailable (ECONNREFUSED 127.0.0.1:6379).

Add the missing v2 config-service mock for the no-ai feed test so it exercises the intended feed-service request instead of failing before config generation. Made-with: Cursor

feat: add no-ai filtering support for my feed

a0eb22e

Merge curated AI tag and title-word filters into My Feed requests so the frontend can ship the experiment without persisting user feed settings. Made-with: Cursor

nimrodkra mentioned this pull request Apr 1, 2026

feat: add a no-ai toggle to my feed dailydotdev/apps#5832

Merged

3 tasks

fix: address no-ai feed review feedback

9841a34

Improve the local filter path and add coverage so the experiment behaves closer to production while keeping the strict no-AI blocking posture. Made-with: Cursor

nimrodkra requested review from idoshamun and pnvasanth April 1, 2026 08:53

idoshamun reviewed Apr 1, 2026

View reviewed changes

idoshamun requested changes Apr 1, 2026

View reviewed changes

fix: narrow no-ai feed experiment scope

47c9cf6

Remove the anonymous and v1-only backend paths so the no-ai filter stays scoped to My Feed's supported feed-service flow. Made-with: Cursor

fix: mock no-ai feed config fetch in tests

e805896

Add the missing v2 config-service mock for the no-ai feed test so it exercises the intended feed-service request instead of failing before config generation. Made-with: Cursor

nimrodkra requested a review from idoshamun April 1, 2026 10:10

idoshamun approved these changes Apr 1, 2026

View reviewed changes

nimrodkra merged commit d990dc8 into main Apr 1, 2026
11 checks passed

nimrodkra deleted the feat/no-ai-feed-filter branch April 1, 2026 10:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add no-ai filtering support for my feed#3766

feat: add no-ai filtering support for my feed#3766
nimrodkra merged 4 commits intomainfrom
feat/no-ai-feed-filter

nimrodkra commented Apr 1, 2026

Uh oh!

pulumi Bot commented Apr 1, 2026 •

edited

Loading

Root Cause Analysis

Dependency Chain

Risk analysis

Uh oh!

nimrodkra commented Apr 1, 2026

Uh oh!

claude Bot commented Apr 1, 2026 •

edited

Loading

Uh oh!

nimrodkra commented Apr 1, 2026

Uh oh!

idoshamun Apr 1, 2026

Uh oh!

nimrodkra Apr 1, 2026

Uh oh!

idoshamun Apr 1, 2026

Uh oh!

nimrodkra Apr 1, 2026

Uh oh!

nimrodkra commented Apr 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

nimrodkra commented Apr 1, 2026

Summary

Test plan

Uh oh!

pulumi Bot commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✨ Neo Explanation

Root Cause Analysis

Dependency Chain

Risk analysis

Resource Changes

Uh oh!

nimrodkra commented Apr 1, 2026

Uh oh!

claude Bot commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review

Issues

🔴 Performance — 82 individual ILIKE conditions on v1 path

🔴 High false-positive risk — Cursor, RAG, inference, transformer

🟡 False positives — person names

🟡 No tests

🟡 No feature flag — any client can toggle this

🟡 Helper functions defined after use

🟢 Positive notes

Uh oh!

nimrodkra commented Apr 1, 2026

Uh oh!

idoshamun Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

nimrodkra Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

idoshamun Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

nimrodkra Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

nimrodkra commented Apr 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pulumi Bot commented Apr 1, 2026 •

edited

Loading

claude Bot commented Apr 1, 2026 •

edited

Loading

🔴 High false-positive risk — `Cursor`, `RAG`, `inference`, `transformer`